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Consensus  Control  of  Complex  and  Multi-scale  Networks 

George  Yin  and  Le  Yi  Wang,  Wayne  State  University,  Detroit,  Ml  482902,  313-577-2479  (Dept.),  313-577-4715,  gyin@math.wayne.edu, 
lywang@wayne.edu 

The  project  is  motivated  by  the  needs  of  more  advanced  methodology  for  filtering,  control,  coordination,  and  dynamic  reconfiguration  of 
networked  systems.  Development  of  such  systems  is  of  essential  values  for  Army's  mission  to  gather  military  intelligence  information  and  to 
assist  military  operations  in  hostile  encounters.  We  have  worked  on  several  problems  arising  in  modeling,  coordination,  control,  computation, 
and  analysis  of  networked  systems.  The  following  is  a  list  of  tasks  accomplished. 

1.  System  Identification  and  Parameter  Estimation 

1)  Joint  observability  and  sampling  complexity  were  established  in  our  paper  in  one  of  the  flagship  journal  in  control  theory, 

Automatica  [7].  We  extended  Shannon's  sampling  theorem  and  our  recent  results  on  sampling  complexity  to  joint  estimation 
problems.  Observer  design  and  convergence  analysis  are  conducted  for  systems  under  noisy  observations.  It  is  shown  that  our 
algorithms  converge  strongly  with  an  error  bound  of  the  order  0(1/N) . 

2)  Summarizing  our  work  on  identifications  of  systems  with  regular  and  quantized  sensors,  we  developed  large  deviations  estimates  for 
such  identification  problems.  A  SpringerBrief  in  Mathematics  was  published  in  2013  (Ref.  [2]).  Although  there  are  many  excellent 
works  on  large  deviations  and  identifications,  our  work  appears  to  be  the  first  one  putting  the  large  deviations  framework  for 
identification  problems.  Observations  under  regular,  binary,  and  quantized  sensors  have  been  considered.  Complexity  issues  are  also 
treated. 

3)  We  studied  identification  of  linear  systems  with  binary-valued  observations  generated  via  fixed  thresholds.  In  addition  to  stochastic 
noises,  the  systems  are  also  subject  to  many  structural  uncertainties,  including  deterministic  unmodeled  dynamics,  nonlinear  model 
mismatch,  and  sensor  observation  bias.  Since  binary-valued  observations  can  supply  only  limited  information  on  the  signals, 
truncated  empirical  measures  were  introduced  to  extract  further  information  for  system  identification.  An  effective  identification 
algorithm  was  constructed  based  on  the  proposed  empirical  measures.  Optimal  identification  errors,  time  complexity,  optimal  input 
design,  and  impact  of  disturbances,  unmodeled  dynamics,  observation  bias,  and  nonlinear  model  mismatch  are  thoroughly 
investigated  in  a  stochastic  information  framework.  Asymptotic  upper  and  lower  bounds  are  established  on  identification  errors  [13]. 


4)  Estimation  of  states  and  events  in  randomly  switching  systems  is  studied  under  irregular  and  random  sampling  schemes. 

Probabilistic  characterization  of  observability  is  presented  under  various  sampling  schemes  and  regime-switching  processes.  The 
characterization  is  derived  on  the  basis  of  our  recent  results  on  sampling  complexity  for  system  observability.  Observer  design  and 
algorithms  are  developed  [6]. 

5)  The  paper  [16]  shows  that  the  matrix  fraction  description,  given  by  a  pair  {A(z),  B(z)}  of  matrix  polynomials  of  z,  for  a  linear  time- 
invariant  system  may  not  be  unique  even  if  A(z)  is  monic,  A(z),  and  B(z)  have  no  common  left  factor,  and  the  matrix  coefficients 
corresponding  to  the  highest-order  terms  of  A(z)  and  B(z)  are  full  row  rank.  The  orders  of  all  possible  MFDs  of  a  given  system  are 
completely  characterized.  Testing  criteria  for  determining  whether  a  matrix  pair  is  an  MFD  of  the  system  are  derived,  which  involve 
rank  tests  of  certain  Toeplitz  matrices  derived  from  either  the  impulse  response  or  output  correlation  functions  of  the  system.  A 
decision  procedure  is  devised  that  generates  sequentially  all  MFDs  for  a  given  system.  Identification  algorithms  are  introduced  that 
estimate  all  MFDs  of  a  given  system  from  its  input-output  data  or  output  data  only.  The  results  are  then  extended  to  cover  ARMAX 
systems. 

6)  The  papers  [17]  and  [18]  introduce  identification  algorithms  for  finite  impulse  response  systems  under  quantized  output 
observations  and  general  quantized  inputs.  While  asymptotically  efficient  algorithms  for  quantized  identification  under  periodic 
inputs  are  available,  their  counterpart  under  general  inputs  has  encountered  technical  difficulties  and  evaded  satisfactory 
resolutions.  Under  quantized  inputs,  this  paper  resolves  this  issue  with  constructive  solutions.  A  two-step  algorithm  is  developed, 
which  demonstrates  desired  convergence  properties  including  strong  convergence,  mean-square  convergence,  convergence  rates, 
asymptotic  normality,  and  asymptotical  efficiency  in  terms  of  the  Cramer-Rao  lower  bound.  Some  essential  conditions  on  input 
excitation  are  derived  that  ensure  identifiability  and  convergence.  It  is  shown  that  by  a  suitable  selection  of  the  algorithm's 
weighting  matrix,  the  estimates  become  asymptotically  efficient.  The  strong  and  mean-square  convergence  rates  are  obtained. 
Optimal  input  design  is  given.  Also  the  joint  identification  of  noise  distribution  functions  and  system  parameters  is  investigated. 
Numerical  examples  are  included  to  illustrate  the  main  results  of  this  paper. 

2.  Stochastic  Systems,  Systems  with  Delays,  and  Systems  with  Random  Switching 

1)  Continuing  on  our  work  [J.  Differential  Eqs.,  2012],  we  developed  moment  exponential  stability  of  stochastic  functional  equations 
(SFDEs)  with  Markovian  switching,  in  which  a  two-time  scale  continuous-time  Markov  chain  is  used  to  represent  the  switching 
process.  Under  suitable  conditions,  there  is  a  limit  system.  Using  the  limit  system  as  a  bridge,  we  establish  a  Razumikhin-type 
theorem  on  moment  exponential  stability.  By  virtue  of  the  Razumikhin-type  theorem,  we  further  deduce  mean-square  exponential 
stability  results  for  stochastic  Volterra  delay-integro-differential  equations  (SVDIDEs),  stochastic  delay  differential  equations  (SDDEs), 
and  stochastic  differential  equations  (SDEs)  with  two-time-scale  Markovian  switching  (see  [15]). 


2)  As  another  effort  of  treating  systems  with  delays,  we  considered  different  types  of  environmental  noises  in  population  systems. 
These  noises  play  crucial  roles  and  have  significant  impact  on  the  evolution  and  biodiversity.  To  understand  the  effects  of  different 
types  of  noises  on  the  asymptotic  properties  of  the  populations,  we  studied  the  influence  of  inherent  net  birth  noise  and  the 
interaction  noise  among  stochastically  perturbed  population  systems  (see  [1]). 

3)  We  developed  almost  sure  stability  criteria  for  switching-jump-diffusion  systems  with  state-dependent  switching.  By  means  of 
introducing  certain  auxiliary  Markov  chains  and  constructing  order-preserving  couplings,  upper  and  lower  "stability  envelops"  were 
constructed,  which  lead  to  systems  with  "upper  and  lower"  approximating  Markov  chains.  Using  these  approximations,  sufficient 
conditions  that  are  relatively  easily  verifiable  for  the  almost  sure  stability  and  instability  were  obtained.  When  the  jump  process  is 
missing,  it  is  demonstrated  that  the  techniques  work  equally  well  and  provide  a  way  to  analyze  the  corresponding  switching 
diffusion  systems  with  x-dependent  switching.  In  addition,  stochastic  stabilization  and  destabilization  are  examined  (Ref.  [11]). 

4)  The  paper  [22]  establishes  ergodic  properties  for  Markovian  semigroups  generated  by  segment  processes  associated  with  several 
classes  of  retarded  stochastic  differential  equations  (SDEs)  with  constant/variable/distributed  time  lags.  It  derives  exponential 
ergodicity  for  (a)  retarded  SDEs  by  the  Arzela-Ascoli  tightness  characterization  of  the  space  C  equipped  with  the  uniform  topology, 
(b)  neutral  SDEs  with  continuous  sample  paths  by  a  generalized  Razumikhin-type  argument  and  a  stability-in-distribution  approach, 
and  (c)  retarded  SDEs  driven  by  jump  processes  using  the  Kurtz  criterion  of  tightness  for  the  space  D  endowed  with  the  Skorohod 
topology. 

3.  Stochastic  Algorithms  for  Consensus,  Swarming,  and  Adaptive  Filtering 

1)  Recently,  much  progress  has  been  made  on  particle  swarm  optimization  (PSO).  A  number  of  works  have  been  devoted  to  analyzing 
the  convergence  of  the  underlying  algorithms.  Nevertheless,  in  most  cases,  rather  simplified  hypotheses  are  used.  For  example,  it 
often  assumes  that  the  swarm  has  only  one  particle.  In  addition,  more  often  than  not,  the  variables  and  the  points  of  attraction  are 
assumed  to  remain  constant  throughout  the  optimization  process.  In  reality,  such  assumptions  are  often  violated.  Moreover,  there 
are  no  rigorous  rates  of  convergence  results  available  to  date  for  the  particle  swarm,  to  the  best  of  our  knowledge.  In  this  paper,  we 
consider  a  general  form  of  PSO  algorithms,  and  analyze  asymptotic  properties  of  the  algorithms  using  stochastic  approximation 
methods.  We  rewrite  the  PSO  procedure  as  a  stochastic  approximation  type  iterative  algorithm.  Then  we  analyze  its  convergence 
using  weak  convergence  method.  It  is  proved  that  a  suitably  scaled  sequence  of  swarms  converge  to  the  solution  of  an  ordinary 
differential  equation.  We  also  establish  certain  stability  results.  Moreover,  convergence  rates  are  ascertained  by  using  weak 
convergence  method.  A  centered  and  scaled  sequence  of  the  estimation  errors  is  shown  to  have  a  diffusion  limit.  The  paper  was 
published  in  another  flagship  journal  IEEE  Transactions  on  Automatic  Control  in  2015  [19]. 


2)  We  developed  a  post-iteration  averaging  algorithm  to  achieve  asymptotic  optimality  in  convergence  rates  of  stochastic 
approximation  algorithms  for  consensus  control  with  structural  constraints.  The  algorithm  involves  two  stages.  The  first  stage  is  a 
coarse  approximation  obtained  using  a  sequence  of  large  stepsizes.  Then  the  second  stage  provides  a  refinement  by  averaging  the 
iterates  from  the  first  stage.  We  show  that  the  new  algorithm  is  asymptotically  efficient  and  gives  the  optimal  convergence  rates  in 
the  sense  of  the  best  scaling  factor  and  "smallest"  possible  asymptotic  variance.  (Ref.  [5]). 

3)  Motivated  by  reduction  of  computational  complexity,  we  develop  sign-error  adaptive  filtering  algorithms  for  estimating  time-varying 
system  parameters  in  [3].  Different  from  the  previous  work  on  sign-error  algorithms,  the  parameters  are  time-varying  and  their 
dynamics  are  modeled  by  a  discrete-time  Markov  chain.  A  distinctive  feature  of  the  algorithms  is  the  multi-time-scale  framework  for 
characterizing  parameter  variations  and  algorithm  updating  speeds.  This  is  realized  by  considering  the  stepsize  of  the  estimation 
algorithms  and  a  scaling  parameter  that  defines  the  transition  rate  of  the  Markov  jump  process.  Depending  on  the  relative  time 
scales  of  these  two  processes,  suitably  scaled  sequences  of  the  estimates  are  shown  to  converge  to  either  an  ordinary  differential 
equation,  or  a  set  of  ordinary  differential  equations  modulated  by  random  switching,  or  a  stochastic  differential  equation,  or 
stochastic  differential  equations  with  random  switching.  Using  weak  convergence  methods,  convergence  and  rates  of  convergence 
of  the  algorithms  are  obtained  for  all  these  cases;  see  [12,  20]. 

4.  Stochastic  Systems  and  Control 

1)  In  many  applications,  one  needs  to  evaluate  a  path-dependent  objective  functional  V  associated  with  a  continuous-time  stochastic 
process  X.  Due  to  the  nonlinearity  and  possible  lack  of  Markovian  property,  more  often  than  not,  V  cannot  be  evaluated  in  closed  form, 
and  only  Monte  Carlo  simulation  or  numerical  approximation  is  possible.  In  addition,  the  calculations  often  require  the  handling  of 
stopping  times,  the  usual  dynamic  programming  approach  may  fall  apart,  and  the  continuity  of  the  functional  becomes  the  main  issue. 
Our  work  published  in  SIAM  J.  Control  and  Optimization  (another  flagship  journal  in  control  theory)  developed  a  numerical  scheme  so 
that  an  approximating  sequence  of  path-dependent  functionals  converges  to  V;  see  [8]. 

2)  As  a  continuation  of  our  work  [Yin  and  Zhang,  1998,  Springer]  on  a  comprehensive  treatment  of  two-time-scale  Markov  chains  with 
finite  state  spaces,  the  monograph  [9]  focuses  on  applications  that  were  considered  after  the  publication  of  the  aforementioned  book. 

In  addition,  countable  state  space  cases  and  switching  diffusion  limits  are  also  treated.  After  reviewing  previous  results  on  two-time- 
scale  Markov  chains  with  a  finite  state  space,  systems  having  countable  state  spaces  are  treated,  and  switching  diffusion  limits  are 
discussed  as  well.  In  addition,  state-dependent  generators  are  dealt  with,  which  largely  extends  the  applicability  of  the  results.  The 
second  part  of  the  book  is  devoted  to  a  number  of  applications  arising  from  manufacturing,  queueing  systems,  financial  engineering. 


insurance  risk  etc.  Hopefully,  these  results  will  be  of  interests  to  many  people  working  in  the  related  areas  and  an  even  wider  range  of 
applications.  Devoted  to  large  scale  and  complex  systems,  it  encompasses  such  applications  as  insurance  and  risk  management,  financial 
engineering,  queueing  networks,  and  the  Wonham  filtering  among  others.  With  seemingly  diverse  range  of  applications,  the  different 
problems  are  closely  connected  through  the  central  theme  of  two-time-scale  formulation.  We  hope  that  this  work  can  serve  as  a  user 
guide  for  modeling,  analysis,  optimization,  and  computation  for  a  wide  variety  of  Markovian  systems;  see  [3]. 

3)  Traditional  stochastic  control  problems  in  the  infinite  horizon  are  concerned  with  the  well-known  discounted  criterion  and  the 
average  criterion  (or  ergodic  criterion).  It  is  widely  recognized  that  these  two  basic  criteria  have  certain  deficiencies.  The  first  one 
concentrates  on  the  performance  of  early  period  of  time  since  the  reward  (or  cost)  has  a  diminishing  weight  for  long-time  intervals, 
whereas  the  second  one  focuses  on  asymptotic  behavior,  but  it  does  not  take  into  account  the  behavior  on  finite  intervals.  As  a  result, 
each  of  these  two  criteria  has  its  limitations.  The  first  one  overlooks,  in  some  sense,  any  large-time  behavior,  whereas  the  second  one 
cannot  provide  any  insight  into  finite  horizon  performance.  To  overcome  the  difficulties  and  drawbacks,  the  so-called  selective  or 
advanced  criteria  have  been  invented.  The  book  focuses  on  the  so-called  advanced  or  selective  control  criteria  for  switching  diffusion 
systems.  Since  the  initiation  of  the  study  of  stochastic  control,  there  have  been  numerous  papers  on  infinite  horizon  control  systems. 
Nevertheless,  the  results  on  advanced  criteria  are  still  scarce.  There  have  been  only  a  handful  of  papers  focusing  on  advanced  criteria. 
Moreover,  most  of  the  existing  papers  on  advanced  criteria  have  focused  on  Markov  decision  processes,  continuous-time  Markov  chains, 
and  controlled  diffusions.  The  work  on  that  of  Markov-modulated  diffusions  is  still  in  its  infancy.  Facing  the  need,  the  monograph  [4] 
takes  up  the  issue  of  studying  advanced  criteria  for  controlled  switching  diffusions.  We  survey  some  of  the  recent  progress  and 
substantially  update  and  extend  the  known  results. 

4)  The  work  [21]  focuses  on  gene  regulatory  networks  driven  by  intrinsic  noise  with  two-time  scales.  It  uses  a  stochastic  averaging 
approach  for  these  systems  to  reduce  complexity.  Comparing  with  the  traditional  quasi-steady-state  hypothesis  (QSSH),  our 
approach  uses  stochastic  averaging  principle  to  treat  the  intrinsic  noise  coming  from  both  the  fast-changing  variables  and  the  slow- 
changing  variables,  which  yields  a  more  precise  description  of  the  underlying  systems.  To  provide  further  insight,  this  paper  also 
investigates  a  prototypical  two-component  activator-repressor  genetic  circuit  model  as  an  example.  If  all  the  protein  productions 
were  linear,  these  two  methods  would  yield  the  same  reduction  result.  However,  if  one  of  the  protein  productions  is  nonlinear,  the 
stochastic  averaging  principle  leads  to  a  different  reduction  result  from  that  of  the  traditional  QSSH. 
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